A Generative Approach for Script Event Prediction via Contrastive Fine-Tuning

Script event prediction aims to predict the subsequent given context. This requires capability infer correlations between events. Recent works have attempted improve correlation reasoning by using pretrained language models and incorporating external knowledge (e.g., discourse relations). Though promising results been achieved, some challenges still remain. First, adopted current ignore event-level knowledge, resulting in an inability capture events well. Second, modeling with relations is limited because it can only explicit markers, cannot many implicit correlations. To this end, we propose a novel generative approach for task, which model fine-tuned event-centric pretraining objective predicts next within paradigm. Specifically, first introduce blank infilling strategy as learning inject into model, then design likelihood-based contrastive loss fine-tuning model. Instead of additional layer, perform sequence likelihoods generated Our soft way without any knowledge. The eliminates need use networks make predictions somewhat interpretable since scores each word event. Experimental on multi-choice narrative cloze (MCNC) task demonstrate that our achieves better than other state-of-the-art baselines. code will be available at https://github.com/zhufq00/mcnc.